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Summary 



This report describes the evaluation of two video programs from Solve It a sixth-grade 
mathematics series under development by the Agency for Instructional Technology in 
Bloommgton, Indiana. The 18-part series, available in September 1987, is designed to 
improve students' critical thinking and problem- solving skills in mathematics. The 
purpose of the evaluation was to determine the instructional efifectiveness and appeal of 
two rough-cut programs, "Geometry and Measurement: Measuring Volume** and "Prob- 
lem Solving: Drawing and Ir.terpreting Tables and Diagrams.- To this end, the pro- 
grams were shown to fifth- and sixth-grade students at nine schools in Illinois, Indi- 
ana. New Jersey, and New Mexico. In nearly all cases, evaluators obtained both student 
and teacher comments and scores from tests given before and after viewing. 

Reviewers found "Measuring Volume** to be both instructlonally effective and enjoyable 
to watch. At several schools, students commented that the program made learning 
about volume more interesting. Students Uked the fact that the characters were near 
their own ages. Both students and teachers enjoyed the "reaMife** example that ex- 
plored the increasing volume of Utah's Great Salt Lake. 

Students had a hard time understanding certain parts of "Drawing and Interpreting 
Tables and Diagrams." In the program, students in a school photography club discover 
the advantages of using tables to determine optimal shutter speeds and aperture set- 
tings. Teachers and students found the information too technical to grasp in such a 
short time (i.e., in the context of a 15-mlnute program). Similarly, students found the 
"real-life" segment on renovation of the Salt Lake City Courthouse to be overly techni- 
cal and not very interesting. 



Background 



Solve It is the sixth-grade component of the elementary mathematics video series begun 
with It Figures and Moth Works. The latter two series, developed by the Agency for In- 
structional Technology (ATT) and consortia of state and provincial education agencies, 
are currently in use throughout the United States and Canada. 



The elementary mathematics projects grew out of recommendations made by the Na- 
tional Council of Super/isors of Mathematics (NCSM) and the National Council of 
Teachers of Mathematics (NCTM). The following three concenxs were dee ned critical. 

1. Problem solving must be the focus of school mathematics in the 1980s. 

2. The concept of basic skills in mathematics must encompass more than com- 
putational facility. 

a. Mathematics programs must take full advantage of the power of calculators 
and computers at all grade levels. 

Both It Figures (fourth-grade) and Math Works (fifth-grade) have as their central goal 
the development of particular problem-solving skills. These skills include using a 
table, drawing a picture of a proble: i, using mental computation, using estimation, and 
recugnlzlng necessaiy informatlor. The application of problem-solving skills to real- 
life situations is the main emphasis of the instructional materials in It Figures, Math 
Works, and now in the sixth-grade project. Solve It 

Each of Solve Its 18 programs illustrates different techniques and skills for problem 
solving. The programs are designed to cover the range of mathematical skills repre- 
sented in the most recent NCSM list.* Specifically, the programs guide students by 
showlTig first a real-life situation in which some problem is present, and then demon- 
strating appropriate techniques for finding a solution. 



Methodology 



Formative evaluation of individual programs from Solve It began in August 1986 when 
the first scripts were sent to the field for review. To date, scripts for 17 of the 18 pro- 
grams have been evaluated by consortium representatives, content experts, and 
teachers. 

Near the end of January 1987, rough-cut versions of **Measuring Volume" and "Drawing 
and Interpreting Tables and Diagrams" were shown to classes in nine schools through- 
out Indiana, Illinois, New Jersey, and New Mexico. A team of two AIT evaluators 
presented the programs in Indiana and Illinois schools. Evaluations in New Jersey and 

NatlonalCouncilofSupcnnsors of Mathematics. •Position Paper on Baalc Mathematical Skllb*, 1976. Re- 
printed In Arithmetic Teacher, October 1977, 25:18-22; Mathematics Teacher, February 1978, 71: 147- 152. 



New Mexico were conducted by ITV coordinators and assistants. (A list of participant 
schools appears in Appendix A.) 

Each evaluation involved 1) a pretest, 2) ore-viewing discussion, 3) viewing of the pro- 
gram, 4) a posttest, and 5) classroom discu^^sion. The pretest was given to students by 
their teachers three to five days in advance. Questions were based on concepts and in- 
formation from the program. (A copy of each pretest /posttest appears in Appendices B 
and C. Pretest and posttest scores are listed m Appendix D). 

The purpose of pre-viewing discussion was to get students to feel comfortable talking 
Students were asked what sorts of television shows they enjoy, what they think of in- 
structional television they have seen, and so forth. During the discussion, evaluators 
asked students to be *TV critics'* and to be honest in their reactions. 

Each of the two Solve It programs was approximately 15 minutes long. During viewing, 
evaluators tried to make themselves inconspicuous while noting students' reactions 
(e.g., inattention and boredom). After each program, the posttest was administered. 

Next, evaluators discussed the program with students. Several general questions were 
considered: "What did you like about the program?" "What didn't you like about the 
program?" "Did you like the characters?" *Was the program interesting?" "Do you 
think the program helped you learn about the intended topic?" Students were encour- 
aged to elaborate on their responses. 



Comments on 'Measuring Volume' 



The majority of students liked this program very much. Many commented that they 
could identify with the main characters, who seemed to be near their own ages. Some 
liked that the program was "up-to-date" (i.e., contemporary clothes, etc.). Others said 
they thought the htmior was good. 

A number of students thought "Measuring Volume" made studjrlng volume more inter- 
esting. Some comments were, "I liked it very much... .It made the subject more inter- 
esting": "I liked the film and think that people who don't know what volume is will 
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learn from the program"; ''I thought.. .it was easy to understand and it had different 
ways of showing how to use volume." 

Students were asked to comment specifically on the different segments. In the first 
story, the main character needs to buy a new aquarium and is tiytng to decide how big it 
should be. She must compare the volume of two aquariums, ea^h with a different shape. 
The second story illustrates the concept of cubic units. A young person working in her 
father's sporting goods ston has to deiermlne how many cases of baseballs to buy to fill 
a certain amount of shelf space. Her solution is to measure the amount of space in 
terms of "cubic baseball boxes." In the "real-lffe" segment, a narrator considers the 
problem of the steadily Incieasing volume of Utah's Great Salt Lake. 

In general students liked all three segments. They especially enjoyed the Great Salt 
Lake and baseball box examples. Students commented that the illustrations approach 
volume in slightly different ways, helping them understand how volume is a measure of 
space. The Great Salt Lake example was appealing to students because it included topics 
that normally would not be presented in a math lesson {e.g., how the volume of the lake 
changes over time, problems created by the lake's increasing volume). 

After viewing, several different classes responded with the correct formula for volume 
(length X width x height). Most classes responded correctly on the posttest that volume 
is a measure of space. One student recalled during post-viewing discussion that volume 
is measured in cubic units. 

The dialogue at the beginning of the program seemed artificial to some students, "like 
the actors were Just reading their lines." One boy said it was unrealistic that the boy in 
the baseball segment "Just happened" to be carrying a calculator with him. According 
to one class, the hosts were "stiff." Another class said the hosts "talked too fast." 

The only other negative reaction concerned the mechanics of the production. Even 
though students were reminded that the program was in rough form (i.e., numbers on 
the screen, extraneous noises and comments, no music, etc.), they still were bothered by 
it. In particular, they found the blank screens for graphics Inserts distracting. 



Comments on 'Drawing and Interpreting Tables 

and Diagrams' 



Students and teachers did not like this program as much as **Measuring Volume." They 
did like the characters and one of the segments, but found two of the segments overly 
technical. 

One story line, divided into two parts, runs through this program. A school photogra- 
phy club has received a donation of photography equipment. The students have every- 
thing they need to set up a darkroom and start taking pictures; however, not one of the 
students knows anything about photography. The teacher suggests that they work in 
two teams, one to set up the dark room, the other to learn about taking pictures. The 
first team discovers that making a diagram will help them set up the darkroom. The 
other team learns that, to rake good photographs, il Is useful to make a table lo keep 
track of shutter speeds and aperture sizes. 

In general, students found these examples helpful in showing how and when to use 
tables and diagrams to solve problems. Students said they liked the characters, partic- 
ularly because they seemed to be near their own ages. Both teachers and students liked 
the way the characters worked together to solve problems; students singled out the epi- 
sode in which characters use a diagram to fit a table and counter into a bathroom/dark- 
room. One class said the use of a mathematical table in the shutter speed/aperture ex- 
ample was a good illustration of when to use a table cind how to set it up. 

The idea of shutter speed needed clarification, students said. Moreover, they found 
there simply was not enough time to absorb the information presented in the photogra- 
phy segment (i.e., shutter speed and aperture size and how both of these factors relate to 
taking a good picture). Some students thought more explanation was needed for the 
concepts to be clear, while others were bothered by too much detail." Students said too 
much time was spent on the technical aspects of photogra.phy and not enough on using 
tables and diagrams. 

The documentary segment on the courthouse renovation didn't appeal to students. One 
boy commented, "Windows aren't really very interesting unless you Ye an architect or 
you're an adult building your own home." The discussion in this segment was hard for 
students to understand: many said the example seemed aimed more at adults than at 
young people. They also found the diagrams hard to see. 




Some students didn't understand what was happening at the beginning of the program. 
They commented that it would be better to show more clearly that this was a school 
photography club and that the photography equipment had * een donated. 

Several students seemed confused about the notion of a "'school club** (i.e., a group of stu- 
dents with a common interest led by an interested teacherj. Mr. Sanchez's chaiacter 
seemed "weak** to some students because he did not know anj^ing about photography. 

Because the rough-cut was chopped at the end, students and teachers did not know how 
the last segment ended. In general, viewers said the program would be much more ap- 
pealing when the "finishing touches'* were added (i.e., music, sound, ending, etc.). 



Summary of Pretest/Posttest Results 



As a means of mea ring the instructional effectiveness of the two programs, a pretest/ 
posttest was developed for each (see Appendices B and C). The pretest was given to stu- 
dents several days in advance. Students took the test with no specific instructions 
except that their scores would not go to their teachers and they were not being graded. 
Students were made to understand that they were the critics. 

The test on •'Measuring Volume** had six questions on which scores were based; the test 
on "Drawing and Interpreting Tables and Diagrams'* had nine. Test results for "Mea- 
suring Volume** were significant (see Appendix D). On the pretest, the average score of 
students in eight schools combined was 3.2 out of 6, or about 50 percent. The average on 
the posttest was 4.7 out of 6. or about 80 percent. There also was a significant Jump in 
score in each school— the smallest increase was Just over one point (Leal Grade School), 
and the laigest almost two points (Ridge Avenue Middle School). 

Test results on "Drawing and Interpreting Tables and Diagrams** were inconclusive; in 
individual cases, and as averages, pretest and posttest scores were nearly identical. 
Moreover, students got almost all of the questions correct on the pretest. These n suits 
suggest an innate difficulty in constructing a substantive test on this material. 



Conclusion 



Teachers and students found **Measuring Volume" to he more effective and appealing 
than **Drawing and Interpreting Tables and Diagrams." One primary reason for the dif- 
ference was the documentary segment In each; the Great Salt Lake example Intrigued 
students, but the courthouse renovation segment did not. The story lines in "Measuring 
Volume" were very clear, students saiu, but the vignettes In "Drawing and Interpreting 
Tables and Diagrams" were harder to follow. The technical nature of the dialogue in 
parts of -Drawing and Interpreting Tables and Diagrams" also was hard for students to 
understand. 

Two other general observations apply to both programs. Students liked the characters 
very much (v-ith some reservations about Mr. Sanchez and the heavj^-set boy in "Draw- 
ing and Interpret ng Tables and Diagrams"), particularly because they seemed to be 
near their own ages. All classes felt that both programs would be more appealing after 
the "finishing touches" were added. 



Appendix A 

Participani Schools 



Illinois 

Chenoa Grade wv^iiool. Chenoa 
Jefferson Grade School Charleston 
Leal Grade School, Urbana 



New Jersey 

Newbury School. Howell 

Peter Muschal Elementary School. Bordentown 

Ridge Avenue Middle School, Neptune 



New Mexico 

Hayes Middle School, Albuquerque 
Truman Midd'e School, Albuquerque 



Indiana 

Marlin Elementary School, Bloorolngton* 



* Students at Marlin did not take the pretests and posttests. 



Appendix B 

Pr©t©st/Posttast for 'Measuring Volume' 



School 



Solve It 'Measuring Volume' 
Pretest/Posttest 

Directions: Select the answer that is the best way to answer the question and place the 
corresponding letter in the blank on the left side of the page. 



1. Do you like math? 

a. yes. a lot 

b. yes. a little 

c. no, not very much 

d. no. not at all 



2. Volume is a measure of 



a. space 

b. area 

c. perimeter 

d. diameter 



3. To calculate the area of a rectangle, you need to know its 

a. perimeter 

b. surface 

c. length and width 

d. diameter 



4. Volume is measured in 

a. square units 

b. linear units 

c. cubic units 



5. What is the formula for calculating the volume of a box shape? 

a. length x v^ridth x height 

b. lenth x width 

c. 1/2 X length X height 
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Solve It 'Measuring Volume' 

Pretest/Fosttest 

Page 2 



6. What is the volume of the larger box shape? 




•1 cubic unit 



> 2 units 



2 units 



a. 4 cubic units 

b. 8 cubic units 

c. 16 cubic units 

d. can't tell 



7. How would you compare these aquariums to see which has more 
volume? 




a. Use the formula for calculating the volume of a box shape. 

b. Use a measurlrj^ cup to see which holds the most cubic centimeters. 

c. You cannot couVi>are the volumes of these two aquariums because 
they have different shapes. 



Appendix C 

Tables and Diagrams' 



School 
Grade 



Solve It 'Drawing and interpreiing Tables and Diagrams' 

Pretest/Posttest 

Directions: Select the answer that Is the best way to answer the question and place the 
corresponding letter in the blank on the left side of the page. 



1. ITie information below Is in the form of a 

a. table 

b. diagram 

c. map , , 



• -•-M.M-t — — — — ? 



1 NtllASKA 



— I-O-W-A- 



2. The Information below is in the form of a 

a. table 

b. diagram 

C. map uri tXPf CTAMCT 
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1... . 












4S 











3. The information below is In the form of a 

a. table 

b. diagram 

c. map 



ERIC 
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4, Tables and diagrams are useful because they 



a. make infomiatlon easy to understand 

b. mix different kinds of information together 

c. contain only the information we need 

5. The information that consists mainly of pictures should be put in the 
form of a . 

a. table 

b. diagram 

c. map 

6. The information that consists mainly of many repeated items should 
be put n the form of a . 

a. table 

b. diagram 

c. map 

7. To put all the parts of a model airplane together correctly, we need a 

a. table 

b. diagram 

c. map 

8. To find out the rate for making a long-distance call, we need a 

a. table 

b. diagram 

c. map 

9. V/e t^se tables and diagrams when we . 

a. have no information we need 

b. can put the information we need on a piece of paper 

c. try to solve a problem which is too confusing 

End of Test 
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Appendix D 

Test Score Data 



Tables and Diagrams 







Pretest 






Posttest 






No. of 


Raw 




No. of 


Raw 




Schools 


Students 


Total 


Average 


Students 


Total 


Average 


Newbury 


31 


217 


7.0 


30 


216 


7.2 


Peter Muschal 


23 


188 


8.2 


23 


186 


8.1 


Jefferson 


55 


412 


7.5 


54 


422 


7.8 


Chenoa 


37 


285 


7.7 


36 


277 


7.7 


Leal 


0 


0 


0.0 


0 


0 


0.0 


Truman 


23 


139 


6.0 


21 


117 


5.6 


Ridge Avenue 


26 


202 


7.8 


26 


211 


8.1 


Hayes 


22 


149 


6.8 


23 


173 


7.5 


Totals 


217 


1,592 


7.3 


213 


1,602 


7.5 



Measuring Volume 







Pretest 






Posttest 






No. of 


Raw 




Score 


No. of 


Raw 




Students 


Total 


Average 


Students 


Total 


Average 


Newbury 


31 


90 


2.9 


30 


132 


4.4 


Peter Muschal 


23 


76 


3.3 


23 


114 


5.0 


Jefferson 


0 


0 


0.0 


0 


0 


0.0 


Chenoa 


0 


0 


0.0 


0 


0 


0.0 


Leal 


46 


179 


3.9 


47 


239 


5.1 


Truman 


23 


60 


2.6 


19 


82 


4.3 


Ridge Avenue 


25 


78 


3.1 


25 


127 


5.1 


H^es 


23 


57 


2.5 


25 


107 


4.3 


Totals 


171 


540 


3.2 


169 


801 


4.7 
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